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The Institute of Education Sciences (IES) publishes practice guides in education 

to bring the best available evidence and expertise to bear on the types of challenges 
that cannot currently be addressed by single interventions or programs. Authors of 
practice guides seldom conduct the types of systematic literature searches that are 
the backbone of a meta-analysis, although they take advantage of such work when 
it is already published. Instead, authors use their expertise to identify the most im- 
portant research with respect to their recommendations and conduct a search of 
recent publications to ensure that the research supporting the recommendations 
is up-to-date. 

Unique to IES-sponsored practice guides is that they are subjected to rigorous exter- 
nal peer review through the same office that is responsible for independent review 
of other IES publications. A critical task for peer reviewers of a practice guide is to 
determine whether the evidence cited in support of particular recommendations is 
up-to-date and that studies of similar or better quality that point in a different di- 
rection have not been ignored. Because practice guides depend on the expertise of 
their authors and their group decisionmaking, the content of a practice guide is not 
and should not be viewed as a set of recommendations that in every case depends 
on and flows inevitably from scientific research. 

The goal of this practice guide is to formulate specific and coherent evidence-based 
recommendations for use by educators addressing the challenge of increasing access 
to higher education. The guide provides practical, clear information on critical top- 
ics related to what schools can do to help students navigate the path to college and 
is based on the best available evidence as judged by the panel. Recommendations 
presented in this guide should not be construed to imply that no further research 
is warranted on the effectiveness of particular strategies for increasing access to 
postsecondary education. 
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be reviewed and applied according to the specific needs of the educators and edu- 
cation agencies using it, and with full realization that it represents the judgments 
of the review panel regarding what constitutes sensible practice, based on the re- 
search that was available at the time of publication. This practice guide should be 
used as a tool to assist in decisionmaking rather than as a “cookbook.” Any refer- 
ences within the document to specific education products are illustrative and do 
not imply endorsement of these products to the exclusion of other products that 
are not referenced. 
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Introduction 

Access to higher education remains a chal- 
lenge for many students who face barriers 
to college entry. Low-income students and 
students who are potentially the first in 
their family to attend college have lower 
college enrollment rates than other stu- 
dents. 1 Although academic preparation ac- 
counts for some of these differences, the 
disparities in college-going rates persist 
for these groups of students even when 
controlling for academic preparation. 2 
College access outcomes have important 
economic and social consequences: col- 
lege graduates earn more than those with 
a high school degree and are more active 
in their communities. 3 

This guide is intended to help schools and 
districts develop practices to increase ac- 
cess to higher education. It can be use- 
ful for individuals who work in schools 
and districts in planning and executing 
strategies to improve preparation for, and 
access to, higher education. A panel of 
experts in college access programs and 
strategies and in research methods devel- 
oped the recommendations in this guide. 
The guide contains specific steps on how 
to implement the recommendations that 
are targeted at school- and district-level 
administrators, teachers, counselors, and 
related education staff. The guide also 
indicates the level of research evidence 
demonstrating that each recommended 
practice is effective. 

As with all What Works Clearinghouse 
(WWC) practice guide panels, this panel 
developed recommendations by consult- 
ing research evidence. The evidence that 
the panel considered in developing this 



1. Choy (2002); National Center for Education 
Statistics (2008). 

2. Ellwood and Kane (2000); Smith et al. (1997). 

3. Baum and Ma (2007); Kane and Rouse (1995); 

National Conference on Citizenship (2006); U.S. 
Census Bureau (2002). 



document ranges from experimental eval- 
uations of college access programs to ex- 
pert analyses of college access practices. 
In looking for effective practices, the panel 
paid particular attention to high-qual- 
ity experimental and quasi-experimental 
studies, such as those meeting the criteria 
of the WWC, 4 and to patterns of practices 
that are replicated across programs. 

The research base for this guide was iden- 
tified through a comprehensive search for 
studies evaluating college access inter- 
ventions and practices. An initial search 
for this type of research conducted in the 
United States in the past 20 years (1988- 
2008) yielded more than 500 studies. Of 
these, 99 studies examined college access 
programs or related practices for high 
school students and were eligible for fur- 
ther review because the study design in- 
cluded a comparison group. These studies 
were reviewed by the WWC to determine 
whether they were consistent with WWC 
standards. Of the 99 studies, 16 studies 
met WWC standards with or without res- 
ervations. These 16 studies of 10 differ- 
ent college access programs represent the 
strongest evidence of the effectiveness of 
college access programs. 

To indicate the strength of evidence sup- 
porting each recommendation, the panel 
relied on the WWC standards for determin- 
ing levels of evidence, described below and 
in Table 1. It is important for the reader to 
remember that the level of evidence rating 
is not a judgment by the panel on how ef- 
fective each of these recommended prac- 
tices will be when implemented, nor are 
they a judgment of what prior research 
has to say about their effectiveness. The 
level of evidence ratings reflect the panel’s 
judgment of the quality of the existing 
literature to support a causal claim that 
when these practices have been imple- 
mented in the past, positive effects on stu- 
dent academic outcomes were observed. 



4. http://www.whatworks.ed.gov/ 
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Table 1. Institute of Education Sciences levels of evidence for practice guides 



Strong 


In general, characterization of the evidence for a recommendation as strong requires both 
studies with high internal validity (i.e., studies whose designs can support causal conclusions) 
and studies with high external validity (i.e., studies that in total include enough of the range 
of participants and settings on which the recommendation is focused to support the conclu- 
sion that the results can be generalized to those participants and settings). Strong evidence 
for this practice guide is operationalized as: 

• A systematic review of research that generally meets WWC standards (see http://ies. 
ed.gov/ncee/wwc/) and supports the effectiveness of a program, practice, or approach 
with no contradictory evidence of similar quality; OR 

• Several well-designed, randomized controlled trials or well-designed quasi-experi- 
ments that generally meet WWC standards and support the effectiveness of a program, 
practice, or approach, with no contradictory evidence of similar quality; OR 

• One large, well-designed, randomized controlled, multisite trial that meets WWC stan- 
dards and supports the effectiveness of a program, practice, or approach, with no 
contradictory evidence of similar quality; OR 

• For assessments, evidence of reliability and validity that meets the Standards for 
Educational and Psychological Testing. 3 


Moderate 


In general, characterization of the evidence for a recommendation as moderate requires 
studies with high internal validity but moderate external validity or studies with high 
external validity but moderate internal validity. In other words, moderate evidence is 
derived from studies that support strong causal conclusions but generalization is uncer- 
tain or studies that support the generality of a relationship but the causality is uncertain. 
Moderate evidence for this practice guide is operationalized as: 

• Experiments or quasi-experiments generally meeting WWC standards and supporting 
the effectiveness of a program, practice, or approach with small sample sizes and/ 
or other conditions of implementation or analysis that limit generalizability and no 
contrary evidence; OR 

• Comparison group studies that do not demonstrate equivalence of groups at pretest 
and, therefore, do not meet WWC standards but that (1) consistently show enhanced 
outcomes for participants experiencing a particular program, practice, or approach 
and (2) have no major flaws related to internal validity other than lack of demonstrated 
equivalence at pretest (e.g., only one teacher or one class per condition, unequal 
amounts of instructional time, or highly biased outcome measures); OR 

• Correlational research with strong statistical controls for selection bias and for dis- 
cerning influence of endogenous factors and no contrary evidence; OR 

• For assessments, evidence of reliability that meets the Standards for Educational and 
Psychological Testing 13 but with evidence of validity from samples not adequately rep- 
resentative of the population on which the recommendation is focused. 


Low 


In general, characterization of the evidence for a recommendation as low means that the 
recommendation is based on expert opinion derived from strong findings or theories in 
related areas and/or expert opinion buttressed by direct evidence that does not rise to 
the moderate or strong level. Low evidence is operationalized as evidence not meeting 
the standards for the moderate or strong level. 



a. American Educational Research Association, American Psychological Association, and National Council on 
Measurement in Education (1999). 

b. Ibid. 
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They do not reflect judgments of the rela- 
tive strength of these positive effects or 
the relative importance of the individual 
recommendations. 

A strong rating refers to consistent and 
generalizable evidence that an inter- 
vention strategy or program improves 
outcomes. 5 

A moderate rating refers either to evidence 
from studies that allow strong causal con- 
clusions but cannot be generalized with 
assurance to the population on which a 
recommendation is focused (perhaps be- 
cause the findings have not been widely 
replicated) or to evidence from studies that 
are generalizable but have more causal 
ambiguity than that offered by experi- 
mental designs (e.g., statistical models of 
correlational data or group comparison de- 
signs for which equivalence of the groups 
at pretest is uncertain). 

A low rating refers to evidence from stud- 
ies that do not meet the standards for 
moderate or strong evidence and/or expert 
opinion based on reasonable extrapola- 
tions from research and theory. 

A low level of evidence rating does not 
indicate that the recommendation is any 
less important than other recommenda- 
tions with a strong or moderate rating. 
Rather, it suggests that the panel cannot 
point to a body of research that demon- 
strates its effect on student achievement. 
In some cases, this simply means that the 



5. Following WWC guidelines, improved out- 
comes are indicated by either a positive, statisti- 
cally significant effect or a positive, substantively 
important effect size (i.e., greater than 0.25). See 
the WWC guidelines at http://ies.ed.gov/ncee/ 
wwc/pdf/wwc_versionl_standards.pdf. 



recommended practices would be difficult 
to study in a rigorous, experimental fash- 
ion; in other cases, it means that research- 
ers have not yet studied this practice, or 
that there is weak or conflicting evidence 
of effectiveness. 6 

Three of the five recommendations made 
by the panel received a low evidence rat- 
ing. For example, recommendation 2, 
which describes the use of assessments 
to measure college readiness, was deter- 
mined to have a low level of evidence (see 
Table 2). This means that there are few ex- 
isting studies designed to test, in a discrete 
and valid manner, the causal relation be- 
tween the utilization of assessment mea- 
sures and college going. Nevertheless, the 
authors of this practice guide, based on 
expert judgment and knowledge of prac- 
tice, consider the use of assessment to be a 
critical component of a well-implemented 
strategic plan for increasing access to col- 
lege. Hence, although the level of evidence 
rating is low, the panel has included as- 
sessment as one of the five recommended 
practices. 

Citations in the text refer to studies of 
programs that have implemented vari- 
ous practices. Not all of these programs 
contribute to the level of evidence rating: 
although some of these programs have 
had rigorous evaluations of their impacts, 
others have not. Furthermore, some of the 
programs that have been rigorously evalu- 
ated have found positive effects on college 
access outcomes; others have not. 



6. For more information, see the WWC Frequently 
Asked Questions page for practice guides, http:// 
ies.ed.gov/ncee/wwc/references/idocviewer/ 
doc.aspx?docid=15&tocid=3. 
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The What Works Clearinghouse 
standards and their relevance to 
this guide 

In terms of the levels of evidence indi- 
cated in Table 1, the panel relied on WWC 
evidence standards to assess the quality 
of evidence supporting educational pro- 
grams and practices. The WWC addresses 
evidence for the causal validity of instruc- 
tional programs and practices according to 
WWC standards. Information about these 
standards is available at http://ies.ed.gov/ 
ncee/wwc/references/idocviewer/doc. 
aspx?docid=19&tocid=l. 7 The technical 
quality of each study is rated and placed 
into one of four categories: 

• Meets Evidence Standards for random- 
ized controlled trials and regression 
discontinuity studies that provide the 
strongest evidence of causal validity. 

• Meets Evidence Standards with Res- 
ervations for all quasi-experimental 
studies with no design flaws and ran- 
domized controlled trials that have 
problems with randomization, attri- 
tion, or disruption. 

• Does Not Meet Evidence Standards for 
studies that do not provide strong evi- 
dence of causal validity. 



• Potentially Meets Standards for studies 
that require additional information to 
determine whether they meet evidence 
standards; typically refers to quasi-ex- 
perimental studies that do not provide 
sufficient information to assess base- 
line equivalence. 

Following the recommendations and sug- 
gestions for carrying out the recommen- 
dations, Appendix D presents more in- 
formation on the research evidence that 
supports each recommendation. 

We appreciate the efforts of Jeffrey Max, 
Christina Clark Tuttle, Kristin Hallgren, 
Moira McCullough, and Sarah Wissel, Math- 
ematica Policy Research staff members 
who participated in the panel meetings, 
characterized the research findings, and 
drafted the guide. We also appreciate the 
help of the many WWC reviewers who 
contributed their time and expertise to 
the review process. We also thank Scott 
Cody, Shannon Monahan, and Neil Seftor 
for helpful feedback and reviews of earlier 
versions of this guide. 
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7. Reviews of studies for this practice guide 
applied version 1.0 WWC standards. Interested 
readers can access these standards at http://ies. 
ed.gov/ncee/wwc/references/iDocViewer/Doc. 
aspx?docld=20&tocld=l . 
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